Autonomy overview
✎🔗Liam
This unit introduces some basic concepts ubiquitous in autonomous vehicle navigation.
Basic Building Blocks of Autonomy
✎🔗The minimal basic backbone processing pipeline for autonomous vehicle navigation is shown in Figure 2.1.
For an autonomous vehicle to function, it must achieve some level of performance for all of these components. The level of performance required depends on the task and the required performance. In the remainder of this section, we will discuss some of the most basic options. In the next section we will briefly introduce some of the more advanced options that are used in state-of-the-art autonomous vehicles.
Sensors
✎🔗
Sensor A sensor is a device that or mechanism that is capable of generating a measurement of some external phyiscal quantity
In general, sensors have two major types. Passive sensors generate measurements without affecting the environment that they are measuring. Examples include inertial sensors, odometers, GPS receivers, and cameras. Active sensors emit some form of energy into the environment in order to make a measurement. Examples of this type of sensor include Light Detection And Ranging (LiDAR), Radio Detection And Ranging (RaDAR), and Sound Navigation and Ranging (SoNAR). All of these sensors emit energy (from different spectra) into the environment and then detect some property of the energy that is reflected from the environment (e.g., the time of flight or the phase shift of the signal)
Raw Data Processing
✎🔗The raw data that is input from a sensor needs to be processed in order to become useful and even understandandable to a human.
First, calibration is usually required to convert units, for example from a voltage to a physical quantity. As a simple example consider a thermometer, which measures temperature via an expanding liquid (usually mercury). The calibration is the known mapping from amount of expansion of liquid to temperature. In this case it is a linear mapping and is used to put the markings on the thermometer that make it useful as a sensor.
We will distiguish between two fundamentally types of calibrations.
Intrinsic Calibration An intrinsic calibration is required to determine sensor-specific paramaters that are internal to a specific sensor.
Extrinsic Calibration An extrinsic calibration is required to determine the external configuration of the sensor with respect to some reference frame.
For more information about reference frames check out Unit F-20 - Reference frames
Calibration is very important consideration in robotics. In the field, the most advanced algorithms will fail if sensors are not properly calibrated.
Once we have properly calibrated data in some meaningful units, we often do some preprocessing to reduce the overall size of the data. This is true particularly for sensors that generate a lot of data, like cameras. Rather than deal with every pixel value generated by the camera, we will process an image to generate feature-points of interest. In “classical” computer vision many different feature descriptors have been proposed (Harris, BRIEF, BRISK, SURF, SIFT, etc), and more recently Convolutional Neural Networks (CNNs) are being used to learn these features.
The important property of these features is that they should be as easily to associate as possible across frames. In order to achieve this, the feature descriptors should be invariant to nuissance parameters.
State Estimation
✎🔗Now that we have used our sensors to generate a set of meaningful measurements, we need to combine these measurements together to produce an estimate of the underlying hidden state of the robot and possibly to environment.
State The state $\state_t \in \statesp$ is a sufficient statistic of the environment, i.e. it contains all sufficient information required for the robot to carry out its task in that environment. This can (and usually does) include the configuration of the robot itself.
What variables are maintained in the statespace $\statesp$ depends on the problem at hand. For example we may just be interested in a single robot’s configuration in the plane, in which case $\state_t \equiv \pose_t$. However, in other cases, such as simultaneous localization and mapping, me may also be tracking the map in the state space.
According to Bayesian principles, any system parameters that are not fully known and deterministic should be maintained in the state space.
In general, we do not have direct access to values in $\state$, instead we rely on our (noisy) sensor measurements to tell us something about them, and then we infer the values.
The video is at https://vimeo.com/232324847.
The animation in Figure 2.4 shows the lane following procedure. The output of the state estimator produces the green arrow in the top left pane.
Navigation and Planning
✎🔗
In general we decompose the task of controlling an autonomous vehicle into a series of nested control loops.
The loops are called nested since the output of the outer loop is used as the reference input to the inner loop. An example is shown in Figure 2.5.
If Figure 2.5 is VERY mysterious to you, then you may want to have a quick look in a basic feedback control textbook. For example [21] or [22].
In this case we show three loops. At the outer loop, some goal state is provided. The actual state of the robot is used as the feedback. The controller is the block labeled Navigation and Motion Planning. The job of this block is generate a feasible path from the current state to the goal state. This is executed in configuration space rather than the state space (although these two spaces may happen to be the same they are fundamentally conceptually different.
The video is at https://vimeo.com/232333925.
Control
✎The next inner loop of the nested controller in Figure 2.5 is the Vehicle Controller, which takes as input the reference trajectory generated by the Navigation and Motion Planning block and the current configuration of the robot, and uses the error between the two to generate a control signal.
The most basic feedback control law (See Unit G-27 - Feedback control is called PID (for proportional, integral, derivative) which will be discussed in Unit G-28 - PID Control. For an excellent introduction to this control policy see Figure 2.7.
We will also investigate some more advanced non-linear control policies such as Model Predictive Control, which is an optimization based technique.
Actuation
✎🔗The very innermost control loop deals with actually tracking the correct voltage to be sent to the motors. This is generally executed as close to the hardware level as possible. For example we have a Stepper Motor HAT See the parts list.
Infrastructure and Prior Information
✎🔗In general, we can make the autonomous navigation a simpler one by exploiting existing structure, infastructure, and contextual prior knowledge.
Infrastructure example: Maps or GPS satellites
Structure example: Known color and location of lane markings
Contextual prior knowledge example: Cars tend to follow the Rules of the Road
Advanced Building Blocks of Autonomy
✎🔗The basic building blocks enable static navigation in Duckietown. However, many other components are necessary for more realistic scenarios.
Object Detection
✎🔗
One key requirement is the ability to detect objects in the world such as but not limited to: signs, other robots, people, etc.
SLAM
✎🔗The simultaneous localization and mapping (SLAM) problem involves simultaneously estimating not only the robot state but also the map at the same time, and is a fundamental capability for mobile robotics. In autonomous driving, generally the most common application for SLAM is actual in the map-building task. Once a map is built then it can be pre-loaded and then used for pure localization. A demonstration of this in Duckietown is shown in Figure 2.9.
The video is at https://vimeo.com/232333888.
Other Advanced Topics
✎Other topics that will be covered include:
-
Visual-inertial navigation (VINS)
-
Fleet management and coordination
-
Scene segmentation
-
Deep perception
-
Text recognition